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Abstract 

A  robust  and  general  local  search  matching 
algorithm  is  used  match  fragmentary  horizon 
lines.  A  horizon  line  model  is  extracted  from 
a  rendered  terrain  map  and  is  then  matched 
to  features  extracted  from  CCD  imagery.  Such 
matching  is  one  means  of  automating  vehicle 
orientation  correction:  a  problem  of  practical 
significance  for  the  UGV  program.  Currently, 
final  orientation  correction  relating  vehicle  co¬ 
ordinates  to  terrain  map  coordinates  must  be 
performed  manually.  Results  are  presented  for 
actual  terrain  and  image  data  collected  at  the 
UGV  Demo  C  test  site. 

1  Introduction 

When  a  vehicle  navigating  with  GPS  and  inertial  guid¬ 
ance  stops,  small  errors  in  pointing  angle  lead  to  large  er¬ 
rors  in  pixel  registration  between  imagery  and  stored  ter¬ 
rain  maps.  For  the  Semi-Autonomous  Vehicles  (SSVs) 
developed  for  the  Unmanned  Ground  Vehicle  (UGV) 
Program,  orientation  estimates  can  be  off  by  1  or  more 
degrees  [Ray95b].  The  resulting  uncertainty  precludes 
terrain  guided  visual  search  and  target  recognition. 
Thus,  a  human  must  hand  select  registration  features 
before  these  activities  are  carried  out. 

Here,  local  search  matching  is  tested  as  a  tool  for  au¬ 
tomating  registration.  Local  search  matching  refers  to  a 
body  of  algorithms  we  have  developed  which  find,  with 
high  probability,  the  optimal  correspondence  mapping 
and  geometric  transformation  between  a  model  and  im¬ 
age  data  [BWR90,  Bev93,  BHP95].  To  match  terrain 
and  image  data,  we  anticipate  a  four  step  process: 

•  Render  3D  terrain  using  the  estimated  vehicle  pose. 

•  Extract  matchable  features  from  the  rendered  ter¬ 
rain  and  actual  imagery. 

This  work  was  sponsored  by  the  Advanced  Research 
Projects  Agency  (ARPA)  under  grant  DAAH04-93-G-422, 
monitored  by  the  U.  S.  Army  Research  Office. 
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•  Optimally  match  the  two  sets  of  features. 

•  Use  matched  features  in  place  of  hand  selected  con- 
trol  points  to  correct  the  orientation  estimate  of  the 
vehicle. 

Results  for  steps  1  through  3  are  presented  on  actual 
data  collected  at  the  UGV  Demo  C  test  site  using  the 
SSV’s  CCD  camera.  Step  4,  it  is  assumed,  will  succeed 
if  matches  are  comparable  to  those  selected  by  a  per¬ 
son.  This  assumption  is  a  fair  starting  point,  but  future 
work  should  address  sensitivity  of  orientation  to  errors 
in  image  space  matching. 

2  Related  Work 

In  some  early  terrain  feature  matching,  Clark  [CCE^SO] 
matched  line  segments  representing  dominant  image 
features.  Levitt  [LLCN87]  proposed  a  way  to  select 
salient  landmarks  from  terrain  data  [LLCN87]  for  nav¬ 
igation.  Stein  [Fri92]  uses  panoramic  horizon  curve 
matching  for  vehicle  localization.  Thompson  and  Suther¬ 
land  [THC“^93,  Tho93,  Sut94]  have  built  a  sophisticated 
expert  system  with  a  domain  specific  image  feature  ex¬ 
traction  algorithm  for  abstracting  structural  terrain  de¬ 
scriptions. 

While  most  of  these  works  address  the  general  prob¬ 
lem  of  vehicle  localization  anywhere  on  a  map,  this  paper 
considers  the  much  simpler  problem  of  orientation  cor¬ 
rection.  Our  work  is  distinguished  by:  1)  reliance  upon 
the  known  position  to  reduce  the  problem  to  2D  match¬ 
ing,  2)  use  of  a  robust  and  general  matching  technique 
to  overcome  feature  fragmentation,  and  3)  use  of  narrow 
FOV  imagery. 

3  Step  1:  Terrain  Rendering 

The  5m  digital  elevation  map  (DEM)  for  the  Demo  C 
test  site  was  obtained  from  Lockheed-Martin.  This  site 
was  selected  because  test  imagery  taken  directly  from 
the  SSV  is  available  along  with  ground  truth  indicat¬ 
ing  vehicle  position  and  pointing  angle  relative  to  fixed 
targets  [Ray95a]. 

A  terrain  rendering  system  has  been  developed  using 
Open-GL  which  simulates  the  FOV  of  the  CCD  sensor 


used  on  the  SSV.  A  simple  lighting  model  is  used  and 
terrain  is  rendered  from  positions  at  which  the  vehicle 
actually  acquired  imagery.  The  vehicle  pointing  angle 
is  derived  from  recorded  vehicle  and  target  positions: 
the  targets  are  other  military  vehicles.  Because  target 
ground  truth  is  being  used  to  derive  pointing  angles,  only 
images  with  targets  near  the  image  center  are  used.  Fig¬ 
ures  la  and  lb  show  two  rendered  terrain  images  for 
which  matching  is  tested  below. 

4  Step  2:  Extracting  Features 

The  local  search  algorithm  matches  one  set  of  line  seg¬ 
ments  to  another,  and  for  this  problem  the  model  and 
data  segments  are  extracted  from  the  rendered  and  ac¬ 
tual  images  respectively.  We  use  our  own  implementa¬ 
tion  of  the  Burns  algorithm  [BHR86]  High  frequency 
texture  in  these  scenes  prevents  horizon  features  from 
being  extracted  unless  the  imagery  is  first  smoothed:  a 
7x7  smoothing  kernel  has  been  used  here.  Even  with 
smoothing,  the  horizons  are  still  sometimes  difficult  to 
extract,  and  significant  fragmentation  occurs.  Figures  Ic 
and  Id  show  the  images  themselves  along  with  the  seg¬ 
ments  extracted  by  the  Burns  algorithm. 

5  Step  3:  Matching 

5.1  Review  and  Overview 

A  complete  explanation  of  local  search  matching  appears 
in  Beveridge’s  dissertation  [Bev93]  and  3D  matching  re¬ 
sults  appear  in  [BR95].  A  controlled  performance  anal¬ 
ysis  of  2D  matching  appears  in  [BRG95].  To  briefly  re¬ 
view  the  approach,  an  iterative  generate- and-test  strat¬ 
egy  moves  from  a  randomly  selected  initial  match  to  one 
that  is  locally  optimal.  A  global  least-squares  fitting 
process  always  aligns  model  and  data  for  any  correspon¬ 
dence  tested.  Thus,  global  geometry  implicitly  directs 
search.  A  match  error  takes  account  both  of  spatial  fit 
and  omission:  how  much  model  is  un-matched. 

Search  is  conducted  over  a  space  of  correspondence 
mappings  C:  C  is  the  powerset  of  possibly  matching 
features  S,  Most  other  algorithms  consider  one-to-many 
matches  [Gri90]  while  our  C  includes  many-to-many 
matches.  Without  many-to-many  mappings,  properly 
matching  piecewise  approximations  to  curves  with  non¬ 
coincident  breakpoints  is  impossible.  This  point  is  im¬ 
portant  here  because  horizon  lines  involve  such  non¬ 
coincident  breakpoints. 

While  at  first  the  initialization  of  search  from  ran¬ 
domly  chosen  matches  may  seem  foolish,  it  is  a  strength 
of  the  approach.  By  running  multiple  trials  from  in¬ 
dependently  chosen  initial  matches,  the  probability  of 
seeing  the  best  (or  near  best)  at  least  once  may  be 
made  arbitrarily  high.  Past  experience  has  demon¬ 
strated  100  trials  is  adequate  to  solve  most  difficult  prob- 

^This  version  has  a  simple  single  Glyph  interface  and  is 
publicly  available  from  our  ftp  site:  ftp .  cs .  colostate .  edu, 
/pub/vision 


lems  [BR95,  BRG95].  Another  benefit  of  multiple  trials 
is  the  structure  and  frequency  of  alternative  solutions 
tells  us  much  about  the  difficulty  of  a  particular  prob¬ 
lem. 

5.2  Matching  Results 

Results  for  two  pairs  of  terrain  and  image  features 
are  presented.  Two  constraints  limit  the  space  of  possi¬ 
ble  matches  between  horizon  model  and  image  line  seg¬ 
ments.  The  first  assumes  that  the  horizon  lies  somewhere 
in  a  band  1  /2  the  height  of  the  image  centered  about  the 
true  position.  The  second  assumes  the  relative  orienta¬ 
tion  between  segments  must  be  less  than  17  degrees  for 
them  to  match.  Even  with  these  constraints,  the  sets 
of  potentially  matching  features  were  very  large:  1183 
for  image  1  and  1577  for  image  2.  The  resulting  search 
spaces  C  contain  2^^®^  and  2^^^^  states  respectively. 

To  explore  the  space  of  possible  matches,  500  trials  of 
subset- convergent  local  search  were  run  on  each  prob¬ 
lem.  The  best  match  found  in  each  case  is  shown  super¬ 
imposed  in  black  in  Figures  le  and  If.  In  both  cases, 
visual  inspection  shows  these  to  be  essentially  correct 
matches.  For  images  1  and  2,  the  best  matches  were 
found  in  a  single  trial  with  probabilities  0.056  and  0.036 
respectively.  Based  upon  this  probability,  it  follows  this 
match  may  be  found  with  better  than  95%  confidence 
running  59  and  90  trials  respectively.  Being  conserva¬ 
tive,  100  trials  is  more  than  sufficient. 

These  problems  are  some  of  the  largest,  in  terms  of 
search  space,  for  which  local  search  matching  has  been 
tested.  To  find  these  matches  reliably,  the  current  C 
implementation  running  on  a  Sparc  20  requires  on  the 
order  of  20  minutes  for  image  1  and  an  hour  for  image  2. 
Clearly,  either  some  domain  specific  tuning  or  use  of  par¬ 
allel  hardware  is  required  to  bring  run-times  down.  Both 
of  these  are  very  reasonable  options  for  future  work.  Use 
of  a  better  feature  extraction  algorithm  would  dramati¬ 
cally  simplify  the  combinatorics  and  parallel  local  search 
is  trivial  due  to  the  independence  of  trials. 

One  general  problem  with  horizon  line  matching  is 
starting  to  be  evident  in  Image  2:  ambiguous  horizon 
structure  leads  to  ambiguous  matches.  While  the  best 
match  shown  here  is  correct,  the  model  for  Image  2  is 
essentially  a  radial  curve.  Consequently,  it  can  match 
the  image  data  at  multiple  points.  Slight  changes  in 
the  parameterization  of  the  match  error  uncovered  this 
sensitivity,  resulting  in  shifts  and  scalings  of  the  true 
horizon. 
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